TITLE OF THE INVENTION 
IMAGE PROCESSING METHOD AND APPARATUS, AND COLOR 
CONVERSION TABLE GENERATION METHOD AND APPARATUS 

5 FIELD OF THE INVENTION 

The present invention relates to an image 
processing apparatus and method, a color conversion 
table generation apparatus and method, an image sensing 
apparatus, a computer program, and a computer readable 
10 recording medium and, more particularly, to a technique 
suitably used to execute a color conversion process for 
image data obtained by sensing an object image. 

BACKGROUND OF THE INVENTION 

15 Some conventional image sensing apparatuses can 

internally change color conversion parameters (hue 
levels , saturation levels , gamma tables , contrast 
levels, and the like) to obtain an image of user's 
preference. In general, in such image sensing 

20 apparatus, the user can select a desired one of a 

plurality of parameter sets prepared in advance by a 
manufacturer, and a color conversion process is 
executed using the selected parameter set to obtain an 
image of user's preference (see Japanese Patent 

25 Laid-Open No. 11-187351). 

However, in such conventional image sensing 
apparatus, upon setting the color conversion 
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parameters, the user must actually change color 
conversion parameters to sense a similar scene, and 
must confirm if an image of his or her preference is 
obtained. That is, the setups of the color conversion 
5 parameters force such trial -and- error sequences, 
resulting in very complicated operations . 

Such change process of the color conversion 
parameters in the conventional image sensing apparatus 
suffers the following problems: the degree of freedom 

10 in change is small, and the need for changing only a 
specific color (e.g., a color of sky) cannot be met 
since such change influences all colors to be 
reproduced. For this reason, a setup that the user 
wants cannot always be attained, and it becomes 

15 difficult to obtain an image of user's preference. 

As a method of performing color conversion to 
change the color of an image captured using a digital 
camera or scanner to a preferred color appearance, a 
method of generating a color conversion table (lookup 

20 table) and performing color conversion using that table 
has been proposed. For example, a color expressed on a 
three-dimensional (3D) color space such as an RGB color 
space or the like can undergo color conversion by 
preparing a 3D lookup table. Color conversion using 

2 5 such lookup table allows to freely design color 

conversion parameters, e.g., to convert only a local 
color on the color space. 
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However, a local color can be changed, but then 
the continuity of table data may be lost when 
parameters which change a color extremely are set . For 
example, when certain data in a lookup table is set to 
5 change an input color extremely, surrounding input 
colors change largely. For this reason, the output 
colors become unnatural. 

Under such circumstances of the prior arts, an 
image processing technique that allows to easily obtain 
10 an image of user's preference is demanded. 

For example, it is demanded to automatically set 
color conversion parameters on the basis of images 
before and after an edit process by retouching or the 
like, and to easily realize color conversion of user's 
15 preference . 

Also, it is demanded to automatically set color 
conversion parameters from a pair of colors designated 
by the user, and to easily realize color conversion of 
user ' s preference . 
20 Furthermore, a technique that can change only a 

desired color and colors close to that color is 
demanded . 

Moreover, it is demanded to prevent discontinuity 
of colors in a reproduced image in a color conversion 
25 technique using a color conversion table. 

SUMMARY OF THE INVENTION 
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According to one aspect of the present invention, 
there is provided an image processing apparatus 
comprising: acquisition device which acquires at least 
one color signal value pair indicating color signal 
5 values from an image; determination device which 

determines an image processing parameter on the basis 
of the acquired color signal value pair; registration 
device which registers the image processing parameter 
determined by the determination device; and conversion 

10 device which converts color signal values of an input 
image on the basis of the image processing parameter 
determined by the determination device, and outputs a 
converted image as an output image. 

Furthermore according to another aspect of the 

15 present invention, there is provided an image 

processing method comprising: acquiring at least one 
color signal value pair indicating color signal values 
from an image; determining an image processing 
parameter on the basis of the acquired color signal 

20 value pair; registering the image processing parameter 
determined in the determination step; and converting 
color signal values of an input image on the basis of 
the image processing parameter determined in the 
determination step, and outputting a converted image as 

25 an output image. 

Furthermore according to another aspect of the 
present invention, there is provided an apparatus for 



- 4 - 



generating a color conversion table, comprising: 
storage device which stores first and second images, 
pixel values of which are expressed on an N- dimensional 
color space; generation device which generates an 
5 N-dimensional color conversion table on the basis of 
differences between pixel values of corresponding 
pixels in the first and second images; and adjustment 
device which adjusts generation of table values of the 
color conversion table by the generation device so that 

10 a change amount of a pixel value defined by the color 
conversion table generated by the generation device 
does not exceed a predetermined value. 

Furthermore according to another aspect of the 
present invention, there is provided a method for 

15 generating a color conversion table, comprising: 

generating an N-dimensional color conversion table on 
the basis of differences between pixel values of 
corresponding pixels in first and second images, pixel 
values of which are expressed on an N-dimensional color 

20 space; and adjusting generation of table values of the 
color conversion table in the generating step so that a 
change amount of a pixel value defined by the color 
conversion table generated in the generation step does 
not exceed a predetermined value. 

25 Other features and advantages of the present 

invention will be apparent from the following 
description taken in conjunction with the accompanying 
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drawings, in which like reference characters designate 
the same or similar parts throughout the figures 
thereof . 



5 BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings , which are incorporated 
in and constitute a part of the specification, 
illustrate embodiments of the invention and, together 
with the description, serve to explain the principles 
10 of the invention. 

Fig. 1 is a block diagram showing an example of 
the arrangement of an image sensing apparatus according 
to the first embodiment; 

Fig. 2 is a block diagram for explaining the 
15 process of an image processing unit according to the 
first embodiment; 

Fig. 3 shows a concept of a CCD digital signal 
after A/D conversion according to the first embodiment; 
Fig. 4 shows a concept of a CCD digital signal 
20 after an interpolation process according to the first 
embodiment ; 

Fig. 5 shows a source image and destination image 
according to the first embodiment; 

Fig. 6 is a block diagram for explaining the 
2 5 process of a parameter determination unit according to 
the first embodiment; 

Fig. 7A shows a color conversion list according 
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to the first embodiment; 

Fig, 7B is a graph for explaining a reference 
signal according to the first embodiment; 

Fig. 8A is a flow chart showing a series of 
5 processes using source and destination images; 

Fig. 8B is a flow chart for explaining a color 
conversion list generation process according to the 
first embodiment; 

Fig. 9 is a block diagram for explaining the 
10 process of an image processing unit according to the 
second embodiment; 

Fig. 10 is a view for explaining a method of 
designating a source color and destination color 
according to the third embodiment ; 
15 Fig. 11 is a block diagram for explaining the 

process of a parameter determination unit according to 
the third embodiment ; 

Fig. 12 shows an example of a designated 
conversion color list according to the third 
2 0 embodiment ; 

Fig. 13 is a flow chart for explaining a 
designated conversion color list generation process 
according to the third embodiment ; 

Fig. 14 is a view for explaining another example 
25 of a designation method of a source color and 

destination color according to the third embodiment; 

Fig. 15 is a block diagram showing the 
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arrangement of an information processing apparatus 
which generates a color conversion table (to be 
referred to as a lookup table hereinafter), and 
executes a color conversion process using the lookup 
5 table according to the fourth embodiment; 

Fig. 16 is a block diagram showing the functional 
arrangement of the lookup table generation process and 
color conversion process according to the fourth 
embodiment ; 

10 Fig. 17 is a block diagram showing a detailed 

functional arrangement of a lookup table generation 
module 2102; 

Fig. 18 is a flow chart for explaining the lookup 
table generation process according to the fourth 
15 embodiment; 

Fig. 19 is a flow chart for explaining a 3D color 
conversion process according to the fourth embodiment; 

Fig. 20 is a block diagram showing a detailed 
functional arrangement of a lookup table generation 
20 module 2102 according to the fifth embodiment; 

Fig. 21 is a flow chart for explaining the lookup 
table generation process according to the fifth 
embodiment ; 

Fig. 22 is a block diagram showing the functional 
25 arrangement of the lookup table generation process and 
color conversion process according to the fifth 
embodiment ; 
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Fig. 23 is a block diagram showing a detailed 
functional arrangement of a customize four -dimensional 
(4D) lookup table generation module 2402; and 

Fig. 24 shows a display example of a list table 
5 of image reproduction parameters or color conversion 
lists, which are generated and registered according to 
the first to third embodiments. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
10 Preferred embodiments of the present invention 

will now be described in detail in accordance with the 
accompanying drawings . 
<First Embodiment > 

Fig. 1 is a block diagram showing an example of 
15 the arrangement of an image sensing apparatus to which 
an image processing apparatus of this embodiment is 
applied. Referring to Fig. 1, an image sensing unit 

101 includes a lens system, aperture, and shutter, and 
forms an object image on the surface of a CCD (image 

20 sensing element) 102. Note that a device other than 
CCD, such as CMOS, can be used as the image sensing 
element The image formed on the surface of the CCD 

102 is photoelectrically converted into an analog 
signal, which is sent to an A/D converter 103. The A/D 

25 converter 103 converts the input analog signal into a 
CCD digital signal (input image signal). 

The CCD digital signal obtained by the A/D 
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converter 103 is sent to an image processing unit 104, 
which converts the input signal on the basis of image 
reproduction parameters corresponding to mode 
information set by an image sensing mode setting unit 
5 108, and outputs an output image signal. The output 
image signal undergoes format conversion into a JPEG 
format or the like in a format converter 105. The 
converted image signal is written in an internal memory 
of the image sensing apparatus or an external memory 

10 such as a compact flash® card or the like by an image 
recorder 106. 

The image sensing mode setting unit 108 sets, in 
the image processing unit 104, image reproduction 
parameters corresponding to an image sensing mode 

15 designated by the user from a plurality of image 
sensing modes. When the user customizes image 
reproduction parameters used in the image processing 
unit 104, parameters determined by a parameter 
determination unit 107 are sent to the image processing 

20 unit 104 to change the image reproduction parameters 
set by the image sensing mode setting unit 108. The 
functions of the respective blocks in the image sensing 
apparatus have been explained. 

Note that the image sensing modes such as a 

25 landscape mode, portrait mode, night scene mode, and 
the like set image reproduction parameters that match 
image sensing conditions in respective scenes. These 
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modes have different gamma tables and matrix 
calculation coefficients. In the second embodiment, 
the modes have different lookup table data in addition 
to the above parameters. For example, the respective 
5 modes have the following setup values : a relatively low 
saturation level is set in the portrait mode; 
relatively high contrast and saturation levels are set 
in the landscape mode; and black -controlled parameters 
are set in the night scene mode. In addition, a 
10 customize mode in which the user can arbitrarily 
register at least one or more parameters may be 
prepared . 

The operation of the image processing unit 104 
will be described in detail below. Fig. 2 is a block 

15 diagram showing the functional arrangement of the image 
processing unit 104 in Fig. 1. The flow of the image 
process in the image sensing apparatus of this 
embodiment will be explained below using the block 
diagram of Fig . 2 . 

20 The CCD digital signal output from the A/D 

converter 103 in Fig. 1 is sent to a white balance 
processor 201 to obtain a white balance coefficient, 
which can convert white in an image into a white 
signal, and the color temperature of a light source. 

25 By multiplying the CCD digital signal by the obtained 

white balance coefficient, a white balance process that 
can convert white in an image into a white signal can 
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be done. The CCD digital signal which has undergone 
the white balance process is sent to an edge emphasis 
processor 207 and interpolation processor 202. 

The interpolation processor 202 calculates all 
5 pixel values for respective color components by 

interpolation using pixel values (color signal values) 
at pixel positions R, Gl, G2, and B from a pixel layout 
300 of a single -plate CCD shown in Fig. 3. That is, 
the processor 202 generates R, Gl, G2, and B plane data 
10 400, as shown in Fig. 4. A matrix arithmetic processor 
203 color- converts respective pixels of the plane data 
400 using: 
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for G = (Gl + G2)/2 

15 The CCD digital signal which has undergone the 

matrix arithmetic process in the matrix arithmetic 
processor 203 is then input to a color difference gain 
arithmetic processor 204 to multiply color difference 
signals by gains. This arithmetic operation is 

20 attained using the following equations (2) to (4). 
That is, the signals Rm, Gm, and Bm obtained by 
equation (1) above are converted into signal values 
expressed on a color space defined by signals Y, Cr, 
and Cb by: 
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The signals Cr and Cb are multiplied by gains (gain 
constant = Gl) using: 
Cr 1 = Gl x Cr 

5 Cb' = Gl x Cb (3) 

The signals Y, Cr' (the signal Cr multiplied by the 
gain), and Cb' (the signal Cb multiplied by the gain) 
are further converted into signals Rg, Gg f and Bg by: 
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10 Note that equation (4) is an inverse matrix operation 
of equation (2). 

The CCD digital signal (Rg, Gg, Bg) that has 
undergone the aforementioned color difference gain 
arithmetic process by the color difference gain 
15 arithmetic processor 204 is sent to a gamma processor 
205. The gamma processor 205 data-converts the CCD 
digital signal using: 

Rt = GammaTable[Rg] (5) 
Gt = GammaTable[Gg] (6) 
20 Bt = GammaTable[Bg] (7) 

The CCD digital signal (Rg, Gg, and Bg) that has 
undergone the gamma process is sent to a hue correction 
arithmetic processor 206. In this arithmetic 
operation, the signals Rt, Gt, and Bt are converted 
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into signals Y # Cr, and Cb by: 
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Furthermore, the signals Cr and Cb are corrected by: 
toll H2fr\ 
fb'\ \H12 H2q\Cbl 

5 Then, the signals Y, Cr ? (corrected signal Cr) , and Cb' 
(corrected signal Cb) are converted into signals Rh, 
Gh, and Bh by: 
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Note that equation (10) an inverse matrix operation of 
10 equation (8). In this way, the CCD digital signal (Rh, 
Gh, Bh) that has undergone the hue correction 
arithmetic process in the hue correction arithmetic 
processor 206 is sent to an edge synthesis processor 
208. 

15 On the other hand, the edge emphasis processor 

207 detects an edge on the basis of the CCD digital 
signal which is sent from the white balance processor 
201 after the white balance process, so as to extract 
only an edge signal. The extracted edge signal is 

20 amplified by a gain-up process, and is sent to the edge 
synthesis processor 208. The edge synthesis processor 

208 executes an edge synthesis process by adding the 
edge signal extracted by the edge emphasis processor 
207 to the signals Rh, Gh, and Bh sent from the hue 
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correction arithmetic processor 206. The flow of the 
signal process has been explained. 

When the user does not change any parameters, 
default values of the selected image sensing mode are 
5 used as parameters set by the image sensing mode 

setting unit 108, i.e., parameters Mil, M21, M31,..., 
M33 in equation (1), Gl and G2 in equation (3), and 
Hll, H21, H12, and H22 in equation (9). By contrast, 
when the user changes image reproduction parameters, 

10 these default image reproduction parameter values are 
substituted by those changed and determined by the 
parameter determination unit 107. The parameter 
determination process by the parameter determination 
unit 107 will be described below. 

15 Image data 501 in Fig. 5 (an image with a file 

name Src.bmp in Fig. 5) is conversion source image data 
(to be also referred to as a source image hereinafter) 
described in a BMP file format, and can be an arbitrary 
image prepared by the user. The user retouches this 

20 conversion source image data 501 to attain color 

reproduction of his or her preference. In this case, 
the user retouches to change only a region to be 
converted in place of uniformly converting the entire 
image. Also, parameters to be retouched are not 

25 particularly limited (e.g., a hue level, saturation 
level, or the like may be changed). In this 
embodiment, only a sky portion is converted into a 
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favorite color. Fig. 5 depicts a destination image 
generated by retouching as image data 502 (file name 
Dst.bmp). Note that the image data 501 and 502 have 
the same size. As the source image data 501, sensed 
5 image data obtained by sensing an object image may be 
used, or thumbnail image data recorded on a recording 
medium together with the sensed image data may be used. 

A series of processes using source and 
destination images will be briefly described below 

10 using the flow chart of Fig. 8A. 

The user retouches source image data 501 as a 
conversion source, which is read out from the internal 
memory of the image sensing apparatus or a detachable 
recording medium outside the image sensing apparatus 

15 (step S701), while reviewing it on the display unit of 
the image sensing apparatus (step S702), and saves the 
obtained image as destination image data 502 (e.g., 
file name Dst.bmp) after conversion as another file 
(step S703). The two image data 501 (Src.bmp) and 502 

20 (Dst^bmp) are recorded in the recording medium or 

internal memory of the image sensing apparatus (step 
S704). The user designates an image sensing mode whose 
image reproduction parameters are to be changed, and 
instructs to execute a parameter change process. 

25 Upon detection of the user's parameter change 

instruction, the parameter determination unit 107 in 
Fig. 1 determines parameters on the basis of color 
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signal values of corresponding pixels in the source 
image data 501 and destination image data 502 (step 
S705). Parameters determined in this embodiment are 
Mil, M21 # M31,..., M33 in equation (1), Gl and G2 in 
5 equation (3), and Hll, H21, H12, and H22 in equation 
(9), The parameters of the image sensing mode are 
updated by the determined parameters (step S706), and 
the updated parameters are registered in the internal 
memory of the image sensing apparatus or the detachable 
10 recording medium outside the image sensing apparatus 
(step S707) . 

Fig. 6 is a block diagram showing an example of 
the arrangement in the parameter determination unit 
107. The operations of the parameter determination 

15 unit 107 in step S705 will be explained below using 

Fig. 6. An image data reader 601 in Fig. 6 reads out 
the source image data 501 and destination image data 
502 stored in a recording medium of the image sensing 
apparatus. A color conversion list generator 602 

20 generates a color conversion list 700 from the RGB 
signal values of the source image into those of the 
destination image, as shown in Fig. 7A, on the basis of 
the differences between RGB signal values of pixels at 
identical positions in the readout source image data 

25 501 and destination image data 502. 

This color conversion list 700 indicates how the 
RGB signal values of respective grid points, which are 
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set on the RGB color space, as shown in Fig. 7B, change 
in color conversion from the source image to the 
destination image. In this embodiment, signal values 0 
to 255 (8 bits) on each of the R, G, and B axes are 
5 divided into eight in 32 -step increments to set 729 (= 
9x9x9) grid points on the RGB color space. The RGB 
signal values on the source image side in the color 
conversion list 700 are those of respective grid 
points, and the RGB signal values on the destination 

10 side correspond to their conversion results. The 

generation process of the list 700 executed in step 
S705 in Fig. 8A will be described in detail later using 
the flow chart of Fig. 8B. The list 700 is generated 
using the RGB values of pixels in the source image 

15 having RGB signal values near grid points and 
corresponding pixels in the destination image. 

A parameter generator 603 determines the 
parameters to implement conversion of RGB values 
recorded in the color conversion list 700. 

20 The list generation sequence will be described 

below in accordance with the flow chart shown in 
Fig. 8B. 

Referring to Fig. 8B, reference signal values are 
set first. The reference signal values are RGB signal 
25 values at each grid point, and signals Rsn (n ■ 0 to 
728), Gsn (n = 0 to 728), and Bsn (n = 0 to 728) are 
generated (step S801). Note that the setting method of 
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grid points is not limited to the above method 
(Fig. 7B). 

Then, source image signals are searched for RGB 

signal values near the generated reference signal 

5 values (Rsn # Gsn # Bsn), and the found RGB signal values 

and pixel position are held (step S802). In this 

search, pixels each having RGB signal values, which 

satisfy [signal value difference E < predetermined 

threshold value Th] , are extracted, and their RGB 

10 signal values and pixel positions are held. The signal 

value diff erence E is given by: 

E = ^(Rsn-Rs(x,y)f + (frsn - Gs(x,y)f + (Bsn -Bs(x,y)) 2 (11) 

where x and y are image coordinate values, and A(x, y) 
is the value of a signal A at the x- and y- coordinate 

15 values. The signal value difference E represents the 

distance between the reference signal values (Bsn, Gsn, 
Bsn) on the color space and RGB signal values (Rs, Gs, 
Bs) in the source image. 

It is checked in step S803 if N or more pixels 

20 are held in step S802. If it is determined in step 

S803 that N or more pixels are held in step S802, the 
flow advances to step S805; otherwise, the flow 
advances to step S804. 

In step S805, signal values corresponding to the 

25 reference signal values are calculated in the following 
sequence. Difference signal values dR, dG, and dB from 
the RGB signal values of corresponding pixels in the 
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destination image are calculated using the following 
equations (12) to (14). That is, the differences 
between the RGB signal values of pixels which are found 
and held in step S802 and corresponding pixels in the 
5 destination image are calculated by: 

dR = Rs(x, y) - Rd(x, y) (12) 
dG = Gs(x, y) - Gd(x, y) (13) 
dB = Bs(x, y) - Bd(x, y) (14) 
Then, the average values (average value dRave of 
10 dR, average value dGave of dG, and average value dBave 
of dB) of dR, dG, and dB of all pixels held in step 
S802 are calculated, and how RGB signal values (Rsn, 
Gsn, Bsn) at each grid point position in the source 
image change in the destination image is calculated 
15 using these average values and: 

Rdn = Rsn - dRave (15) 
Gdn = Gsn - dGave (16) 
Bdn = Bsn - dBave (17) 
On the other hand, if N or more pixels each 
20 having the signal value difference E smaller than the 

threshold value Th are not present in the source image, 
it is determined that the pixel is noise, and the 
reference signal values (Rsn, Gsn, Bsn) are directly 
used as RGB signal values (Rdn, Gdn, Bdn) at a 
25 corresponding grid point position in the destination 
image . 

Rdn - Rsn (18) 
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Gdn = Gsn (19) 
Bdn = Bsn (20) 
The aforementioned processes are repeated for all grid 
points (n = 0 to 728). 
5 The parameter generator 603 determines parameters 

to implement color conversion recorded in the color 
conversion list, which is generated in this way. The 
generator 603 executes inverse conversion of equations 
(10) to (1) of RGB signal values (corresponding to 

10 reference signal values (signals Rsn, Gsn, and Bsn) on 
the source image side in the conversion list. In this 
inverse conversion, parameters of the image sensing 
mode to be changed are used. Let Rn, Gn, and Bn be the 
inverse conversion results. Also, let Rdn', Gdn', and 

15 Bdn' be results (signal values) obtained by converting 

these values Rn, Gn, and Bn by equations (1) to (10) 

using the changed parameters. Then, new parameters of 

the image sensing mode of interest are determined using 

the damped- least -square method to minimize a value Diff 

20 given by: 

Diff = ^(Rdn - Rdn ') 2 + (Gdn - Gdn f + (Bdn - Bdn'f (21) 

The parameter generator 603 stores the new 
parameters, which are calculated from the relationship 
between the source and destination images in this way, 
25 in the internal memory of the image sensing apparatus 
or a detachable recording medium (not shown) outside 
the image sensing apparatus as parameters of the image 
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sensing mode, which is designated to change its 
parameters . 

The flow of sensed image data in the 
parameter -changed image sensing mode will be described 
5 below using Figs . 1 and 2 . When the image sensing mode 
whose parameters have been changed by retouching the 
source image, as described above, is selected, and an 
image sensing operation is made, a sensed CCD signal is 
sent from the image sensing unit 101 to the image 

10 processing unit 104 via the A/D converter 103. In the 
image processing unit 104, the processes from the white 
balance processor 201 to the edge synthesis processor 
208 (Fig. 2) are executed. These processes use the 
changed parameters read out from the internal memory of 

15 the image sensing apparatus or the detachable recording 
medium outside the image sensing apparatus in place of 
default parameters prepared in advance. 

Image data which has been processed using the 
changed parameters is sent to the format converter 105, 

20 and is converted in accordance with an image format. 

The converted image data is then written in a recording 
medium of the image sensing apparatus by the image 
recorder 106. The flow of data in the 
parameter-changed image sensing mode has been 

25 explained. 

As described above, according to this embodiment, 
new parameters calculated from the relationship between 
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the source and destination images can be held in a 
recording medium as a parameter -changed image sensing 
mode. By selecting this parameter- changed image 
sensing mode, the changed parameters are applied to the 
5 subsequent image sensing operations, and sensed images 
automatically undergo image processes. For this 
reason, the color appearance of user's preference can 
be reflected upon sensing images, and sensed images 
with a common color appearance can be automatically 

10 obtained by setting the parameters only once. In this 
case, information that can specify the parameters, 
color conversion list, and LUT used to process images 
in the image sensing operation may be recorded in tag 
information or the like together with images . 

15 In the above embodiment, parameters of a 

predetermined image sensing mode are changed, and the 
changed parameters overwrite those of that image 
sensing mode. However, the present invention is not 
limited to this. For example, one or a plurality of 

20 image sensing modes (custom modes) which can be 
customized by the user may be prepared, and the 
generated parameters may be held as those of one of the 
custom modes. In this way, the parameters of the image 
sensing modes prepared in advance can be maintained. 

25 As described above, according to the first 

embodiment, arbitrary source image data and destination 
image data obtained by retouching the source image data 
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are recorded, and when the user designates to change 
image reproduction parameters of a desired image 
sensing mode, a color conversion list is generated by 
comparing the source and destination image data, and 
5 image reproduction parameters in that image sensing 
mode are changed using the generated color conversion 
list. For this reason, favorite color setups of the 
user can be easily given to an image to be reproduced. 
Also, only local colors can be easily changed to those 

10 of user's preference in place of the entire image. 

In the first embodiment, one image sensing mode 
whose image reproduction parameters are to be rewritten 
is designated, and the parameters of only the 
designated image sensing mode are rewritten and 

15 registered. Alternatively, the parameters of a 

plurality of image sensing modes designated by the user 
may be simultaneously rewritten in place of one mode. 
In this case, the inverse conversion of equations (10) 
to (1) is executed using the image reproduction 

20 parameters set in the designated image sensing modes. 
That is, the process of the parameter generator 603 is 
executed for each of the designated image sensing modes 
using one color conversion list. 

In the first embodiment, the BMP file format is 

25 used as the format of the source and destination 
images. However, the format of the source and 
destination images is not limited to such specific 
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format. For example, other file formats such as JPEG, 
TIFF, and the like may be used. 

In the first embodiment, since the image sensing 
apparatus is designed to execute the image processes 
5 including the aforementioned parameter determination 
process, the aforementioned color conversion process 
can be done upon executing an image sensing process . 
However, the image sensing apparatus need not always be 
designed to execute the image processes. For example, 

10 an image processing apparatus that executes the image 
processes may be arranged separately. With this 
arrangement, the aforementioned color conversion 
process can be made by inputting arbitrary image data 
possessed by the user to that apparatus, and favorite 

15 color setups of the user can be easily realized. 

In this case, for example, image conversion 
parameters are determined using source and destination 
images or a color conversion list in place of these 
images , and image reproduction parameters prepared in 

20 advance in the image sensing apparatus are changed 

using the determined parameters. Then, a registration 
operation for registering the generated color 
conversion lists or changed image conversion parameters 
by assigning names to them is executed. In the 

25 registration operation, the color conversion list or 
color conversion parameters may be registered by 
designating an image sensing mode whose image 
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reproduction parameters are to be changed, and 
executing the parameter change process. Alternatively, 
the image processing apparatus can execute color 
conversion of arbitrary image data possessed by the 
5 user using the registered image conversion parameters. 
Note that the image conversion parameters correspond to 
the image reproduction parameters mentioned above. The 
image processing apparatus may be arranged 
independently of the image sensing apparatus. If such 
10 independent arrangement is adopted, the generated image 
reproduction parameters are transferred to the image 
sensing apparatus via an interface such as USB or the 
like. 

< Second Embodiment > 

15 The second embodiment of the present invention 

will be described below. In the first embodiment, 
image reproduction parameters given by equations (1), 
(3), and (9) are changed on the basis of the generated 
color conversion list. In the second embodiment, 

20 lookup table values (grid point data) in a 3D lookup 
table arithmetic processor 909 shown in Fig. 9 are 
changed. Fig. 9 is a block diagram for explaining the 
processes included in the image processing unit 104 
according to the second embodiment. The flow of image 

25 processes in the image sensing apparatus of this 

embodiment will be described below using the block 
diagram of Fig. 9. 
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Since the functions of a white balance processor 
901, interpolation processor 902, matrix arithmetic 
processor 903, color difference gain arithmetic 
processor 904, gamma processor 905, hue correction 
5 arithmetic processor 906, edge emphasis processor 907, 
and edge synthesis processor 908 are the same as those 
in the first embodiment described above, a detailed 
description thereof will be omitted. In this 
embodiment, a 3D lookup table arithmetic processor 909, 

10 which is not equipped in the first embodiment, will be 
described in detail below. 

CCD digital signals (input RGB signals) Rh, Gh, 
and Bh output from the hue correction arithmetic 
processor 906 are sent to the 3D lookup table 

15 arithmetic processor 909, which converts the input 

signals into CCD digital signals (output RGB signals) 
RL, GL, and BL using a 3D lookup table. 

A 3D lookup table arithmetic operation will be 
briefly described below. Note that the following 

20 description will be given using an RGB 3D lookup table. 
However, a multi-dimensional lookup table may be used 
in correspondence with signals to be processed. 

In this embodiment, in order to reduce the size 
of the 3D lookup table, 729 (= 9 x 9 x 9) 3D 

25 representative grid points (lookup table) obtained by 
dividing the range from minimum to maximum values of 
each of R, G, and B signals into nine are prepared, and 
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RGB signals other than those at representative grid 
points are calculated by interpolation. 

The interpolation arithmetic operation is 
described by: 
5 R = Ri + Rf 

G = Gi + Gf 

B = Bi + Bf 

Rout(R,G,B) = Rout(Ri+Rf ,Gi+Gf ,Bi+Bf ) = 

( Rout ( Ri , Gi . Bi ) x ( Step-Rf ) x ( Step-Gf ) x ( Step-Bf ) 

10 +Rout ( Ri+Step , Gi , Bi ) x ( Rf ) x ( Step-Gf ) x ( Step-Bf ) 

+Rout ( Ri , Gi+Step , Bi ) x ( Step-Rf ) x ( Gf ) x ( Step-Bf ) 
+Rout ( Ri , Gi , Bi+Step ) x ( Step-Rf ) x ( Step-Gf ) x ( Bf ) 
+Rout ( Ri+Step , Gi+Step , Bi ) x ( Rf ) x ( Gf ) x ( Step-Bf ) 
+Rout ( Ri+Step , Gi , Bi+Step ) x ( Rf ) x ( Step-Gf ) x ( Bf ) 

15 +Rout ( Ri , Gi+Step , Bi+Step ) x ( Step-Rf ) x ( Gf ) x ( Bf ) 

+Rout ( Ri+Step , Gi+Step , Bi+Step ) x ( Rf ) x ( Gf ) x ( Bf ) ) 
/ (StepxStepxStep) (22) 
Gout(R,G,B) = Gout(Ri+Rf .Gi+Gf ,Bi+Bf ) = 
( Gout ( Ri , Gi , Bi ) x ( Step-Rf ) x ( Step-Gf ) x ( Step-Bf ) 

20 +Gout ( Ri+Step , Gi , Bi ) x ( Rf ) x ( Step-Gf ) x ( Step-Bf ) 

+Gout ( Ri , Gi+Step , Bi ) x ( Step-Rf ) x ( Gf ) x ( Step-Bf ) 
+Gout ( Ri , Gi , Bi+Step ) x ( Step-Rf ) x ( Step-Gf ) x ( Bf ) 
+Gout ( Ri+Step , Gi+Step , Bi ) x ( Rf ) x ( Gf ) x ( Step-Bf ) 
+Gout ( Ri+Step , Gi » Bi+Step ) x ( Rf ) x ( Step-Gf ) x ( Bf ) 

25 +Gout ( Ri , Gi+Step , Bi+Step ) x ( Step-Rf ) x ( Gf ) x ( Bf ) 

+Gout ( Ri+Step , Gi+Step , Bi+Step ) x ( Rf ) x ( Gf ) x ( Bf ) ) 
/(StepxStepxStep) (23) 
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Bout(R,G,B) = Bout(Ri+Rf ,Gi+Gf ,Bi+Bf ) = 
( Bout ( Ri , Gi , Bi ) x ( Step-Rf ) x ( Step-Gf ) x ( Step-Bf ) 
+Bout ( Ri+Step , Gi , Bi ) x ( Rf ) x ( Step-Gf ) x ( Step-Bf ) 
+Bout ( Ri , Gi+Step , Bi ) x ( Step-Rf ) x ( Gf ) x ( Step-Bf ) 
5 +Bout ( Ri , Gi , Bi+Step ) x ( Step-Rf ) x ( Step-Gf ) x ( Bf ) 

+Bout ( Ri+Step , Gi+Step , Bi ) x ( Rf ) x ( Gf ) x ( Step-Bf ) 
+Bout ( Ri+Step , Gi , Bi+Step ) x ( Rf ) x ( Step-Gf ) x ( Bf ) 
+Bout ( Ri , Gi+Step , Bi+Step ) x ( Step-Rf ) x ( Gf ) x ( Bf ) 
+Bout ( Ri+Step , Gi+Step , Bi+Step ) x ( Rf ) x ( Gf ) x ( Bf ) ) 
10 / ( StepxStepxStep ) ( 24 ) 

where R, G, and B are input RGB signals, Rout(R,G,B), 
Gout(R,G,B), and Bout(R,G,B) are output RGB signals at 
that time. Also, Ri, Gi, and Bi are signals at a 
representative grid point, which are smaller than and 
15 nearest to the signal values of the input RGB signals 
R, G, and B. Furthermore, Rout (Ri,Gi,Bi) , 
Gout (Ri,Gi,Bi) , and Bout (Ri,Gi,Bi) are representative 
grid point output signals, and Step is the step width 
(= 32 in this embodiment) at a representative grid 
20 point. 

Assume that these lookup table conversion and 
interpolation formulas (22), (23), and (24) are simply 
described by: 

(Rout, Gout, Bout) = LUT[(R, G, B) ] (25) 
25 where R, G, and B are input signal values, LUT is a 9 x 
9x9 lookup table, and Rout, Gout, and Bout are lookup 
table conversion and interpolation results. 
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Using the aforementioned arithmetic operations, 
input RGB signals Rh, Gh, and Bh are converted into 
output RGB signals RL, GL, and BL by: 

(RL, GL, BL) = LUT[(Rh, Gh, Bh) ] (26) 
5 The signals which have undergone the 3D lookup 

table conversion and interpolation operations are sent 
to the edge synthesis processor 908. The edge 
synthesis processor 908 adds an edge signal extracted 
by the edge emphasis processor 907 to the output RGB 
10 signals (RL, GL, BL) sent from the 3D lookup table 
arithmetic processor 909. The flow of the image 
process arithmetic operations according to the second 
embodiment has been explained. 

When the user does not change any parameters , 
15 default values of image reproduction parameters set in 
advance in correspondence with the selected image 
sensing mode are used as Mil, M21, M31,..., M33 in 
equation (1), Gl and G2 in equation (3), Hll, H21, HI 2, 
and H22 in equation (9), and respective values of the 
20 lookup table LUT of equation (26). 

In the second embodiment , when the user changes 
parameters, lookup table grid point data of equation 
(26) are determined, and only the lookup table grid 
point data are substituted. The parameter conversion 
25 operation will be described below. When the color 
conversion list 700 is generated, lookup table grid 
point data are determined on the basis of the color 
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conversion list 700. In the parameter change process 
of this embodiment, since the same processes as in the 
first embodiment are executed until generation of the 
color conversion list 700, a detailed description 
5 thereof will be omitted. After the color conversion 
list 700 is generated, 3D lookup table grid point data 
are determined on the basis of the color conversion 
list 700. Note that the 3D lookup table grid point 
data are determined using the color conversion list 
10 700, and a default lookup table, which is set in 
advance in the image sensing apparatus. 

Initially, a 3D lookup table LUTlist used to 
convert a source image signal into a destination image 
signal is generated based on the color conversion list 
15 700. The 3D lookup table conversion and interpolation 
processes of this color conversion list are done using: 
(Rdn, Gdn, Bdn) = LUTlist [ (Rsn , Gsn, Bsn)] (27) 
When color conversion based on the color 
conversion list is reflected onto 3D lookup table 
20 conversion of the image sensing apparatus, it is 
described by: 

(RL 1 , GL", BL') = LUTlist [ LUT [ (Rh, Gh, Bh) ] ] 

(28) 

Since the default lookup table and the lookup 
25 table based on the color conversion list can be merged 
into one lookup table, equation (28) can be rewritten 
to a single lookup table given by: 
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(RL', GL', BL 1 ) - LUTcustmf (Rh, Gh, Bh) ] (29) 
The new lookup table LUTcustm obtained by merging 
the two lookup tables (LUT, LUTlist) is substituted as 
the 3D lookup table of the image sensing mode of user's 
5 choice, whose parameters of the image sensing apparatus 
are to be changed. 

As described above, according to the second 
embodiment, arbitrary source image data and destination 
image data obtained by retouching the source image data 

10 are recorded. When the user designates an image 

sensing mode whose parameters are to be changed, and 
also designates to execute the parameter change 
process, a color conversion list is generated by 
comparing the source and destination image data, 3D 

15 lookup table grid point data are determined based on 

the generated color conversion list, and the determined 
3D lookup table grid point data are used as those in 
the designated image sensing mode. Hence, favorite 
color setups of the user can be easily given to an 

20 image to be reproduced. Also, since only the lookup 
table grid points whose colors are to be changed are 
changed, only local colors can be easily changed to 
those of user's preference in place of the entire 
image . 

25 That is, when the image sensing mode is to be 

customized using parameters of the matrix arithmetic 
operation, color difference gain arithmetic operation, 
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gamma process, hue correction arithmetic process, and 
the like as in the first embodiment, if the user wants 
to change only a specific color, it is impossible to 
change only that color. However, when the 3D lookup 
5 table is used as in this embodiment, only a specific 
color can be changed without changing other colors . 

In this embodiment, the source and destination 
images are recorded on the recording medium of the 
image sensing apparatus, which internally generates a 

10 color conversion list and determines parameters based 

on the generated color conversion list, as in the first 
embodiment. Alternatively, a color conversion list may 
be generated in advance using a personal computer, the 
color conversion list may be directly recorded on a 

15 recording medium, and parameters may be determined 
based on that color conversion list. 
< Third Embodiment > 

In the first embodiment, the color conversion 
list 700 is generated using the RGB signal values of 

20 corresponding pixels in images before and after the 
retouch process. In the third embodiment, a color 
conversion list is generated using the RGB signal 
values of a pair of pixels designated by the user. 
Note that the image reproduction parameters described 

25 in the first embodiment can be changed or the lookup 
table described in the second embodiment can be 
generated or updated on the basis of the color 
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conversion list generated in the third embodiment. 

The generation process of the color conversion 
list 700 according to the third embodiment will be 
described below. 
5 Images shown in Fig. 10 include a source image 

1001 prepared to designate a first color as a 
conversion source, and a destination image 1002 
prepared to designate a second color as a conversion 
target color of the first color. The user prepares for 

10 these two images, which may be those sensed by an image 
sensing apparatus different from that of this 
embodiment, or those which are generated by the user by 
retouching. The formats of the source and destination 
images are not particularly limited as long as the 

15 image sensing apparatus of this embodiment is 

compatible. For example, JPEG, TIFF, GIF, and the like 
may be used. The source and destination images may 
have different sizes. Also, the source and destination 
images may use sensed image data obtained upon sensing 

20 an image or thumbnail image data recorded on a 

recording medium together with the sensed image data. 
The sensed image data may be RAW data, which is 
obtained by A/D- converting an output signal from an 
image sensing element and converting digital data to a 

25 predetermined format with or without reversible 
compression. 

The user records the two images in a recording 
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medium of the image sensing apparatus, and starts 
parameter determination. The parameter determination 
process is done by the parameter determination unit 107 
in Fig. 1. Fig. 11 is a block diagram showing the 
5 functional arrangement of the parameter determination 
unit 107 according to the third embodiment. The 
operation of the parameter determination unit 107 
according to the third embodiment will be described 
below using Fig. 11. 

10 An image data reader 1601 reads out the source 

and destination image data stored in a recording medium 
of the image sensing apparatus. The readout image data 
are displayed on an image display unit of the image 
sensing apparatus, although not shown. The user 

15 designates a source color as a conversion source in the 
source image displayed on the image display unit by 
moving cursor A, as shown in Fig. 10. Likewise, the 
user designates a destination color as a conversion 
target of the source color from the destination image 

20 by moving cursor B. If there are a plurality of source 
colors to be converted, the user designates the next 
source color in the source image, and a destination 
color as a conversion target of that source color in 
the destination image. As a color designated by the 

25 cursor A or the cursor B, value of the color of one 
pixel specified with the cursor can be used. 
Alternatively, mean value of the color of the pixel of 
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neighborhood at the position specified with the cursor 
can may be used as the designated color. With this 
color designation operation, a designated conversion 
color list shown in Fig. 12 is generated. The 
5 designated conversion color list is registered with 
pairs of RGB signal values (Rs(i), Gs(i), Bs(i)) of 
source colors and RGB signal values (Rd(i), Gd(i), 
Bd(i) of destination colors designated as conversion 
targets of the source colors. Let C be the number of 

10 colors to be converted set by the user (the number of 
pairs of colors). Then, i = 0 to C - 1 . 

After the designated conversion color list is 
generated in this way, it is sent to a color conversion 
list generator 1603. The color conversion list 

15 generator 1603 generates the color conversion list 700, 
which has been explained in the first embodiment, on 
the basis of the designated conversion color list. A 
parameter generator 1604 updates image reproduction 
parameters (Mil, M21, M31,..., M33, Gl, G2 , Hll, H21, 

20 H12, H22, or lookup table values) by the method 

described in the first or second embodiment on the 
basis of the contents of the generated color conversion 
list 700. 

The sequence for generating parameters based on 
25 the color conversion list will be described below with 
reference to the flow chart of Fig. 13. 

In step S1801, reference signal values Rsn, Gsn, 
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and Bsn are generated. These reference signal values 

are the same as those described in the first embodiment 

with reference to Fig. 7B. In step S1802, the 

designated conversion color list is searched for RGB 

5 signal values near the reference signal values 

generated in step S1801. In this search process, a 

signal value difference E is calculated for each of 

source col ors i = 0 to C - 1 using: 

E = <J(Rs(i)- Rsnf + (Gs(i)- Gsn f + (Bs(i)-Bsnf (30) 

10 Then, RGB signal values which satisfy [signal value 
difference < threshold value Th] are extracted. 

If the RGB signal values of one or more colors 
are extracted in step S1802, the flow advances to step 
S1805 to calculate color signal values corresponding to 
15 the reference signal values after conversion in the 
following sequence. That is, difference signals dR, 
dG, and dB between the RGB signal values of the 
extracted source color and those of a destination color 
recorded in the designated conversion color list as a 
20 pair of that source color are calculated: 

dR = Rs(i) - Rd(i) (31) 
dG = Gs(i) - Gd(i) (32) 
dB = Bs(i) - Bd(i) (33) 
where Rs(i), Gs(i), and Bs(i) are the RGB signal values 
25 of the extracted source color, and Rd(i), Gd(i), and 

Bd(i) are those of a destination color corresponding to 
that source color. 
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The average value dRave of dR, the average value 
dGave of dG, and the average value dBave of dB of 
extracted signals are calculated, and signal values 
Rdn, Gdn, and Bdn after color conversion corresponding 
5 to Rsn, Gsn, and Bsn are calculated by arithmetic 

operations with the reference signal values Rsn, Gsn, 



and Bsn: 

Rdn = R(n) - dRave (34) 

Gdn = G(n) - dGave (35) 

10 Bdn = B(n) - dBave (36) 



If no RGB signal values which satisfy [signal 
value difference E < threshold value Th] are found, the 
flow advances from step S1803 to step S1804 to 
calculate Rdn, Gdn, and Bdn by: 
15 Rdn = Rsn (37) 

Gdn = Gsn (38) 
Bdn = Bsn (39) 
By repeating the aforementioned process for n = 0 
to 728, the color conversion list is generated. Based 
20 on the color conversion list generated in this way, the 
image reproduction parameters or 3D lookup table values 
are generated. A lookup table, which is set in 
advance, may be updated by overwriting it by the 
generated lookup table. 
25 In the third embodiment, upon generating the 

color conversion list, two images, i.e., source and 
destination images are used. Alternatively, as shown 
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in Fig. 14, a pair of source and destination colors may- 
be designated from a single image. Furthermore, a pair 
of source and destination colors may be designated from 
three or more images . The parameters of the 3D lookup 
5 table in this embodiment have 729 grid points in 

32-step increments. However, the present invention is 
not limited to this , and any other grid point intervals 
may be used. 

As described above, according to the third 

10 embodiment, upon setting color reproduction of user's 

preference, since the user designates a source color as 
a conversion source color, and a destination color as a 
target color after conversion, favorite color setups of 
the user of the image sensing apparatus can be easily 

15 realized. Since color conversion is made using the 3D 
lookup table in place of parameters that influence the 
entire image, only local colors can be easily converted 
into favorite colors. 

In the first to third embodiments, source and 

20 destination images expressed by RGB signals have been 
explained. However, the present invention is not 
limited to the RGB signals, and CMYG signals may be 
used. When CMYG signals are used, the above 
embodiments may be modified to use a four-dimensional 

25 lookup table. Also, YCrCb signals, L*a*b* signals, and 
the like may be used. 

In the first to third embodiments, image 
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reproduction parameters generated based on source and 
destination images, or a color conversion list or 
lookup table generated based on the parameters can be 
recorded in the internal memory of the image sensing 
5 apparatus or a detachable recording medium. Upon 

recording, since a plurality of color conversion lists 
and the like may be recorded, it is preferable to 
register these lists by assigning names to them. When 
a plurality of custom modes that can respectively 

10 register different color conversion lists, image 

conversion parameters, or lookup tables are prepared, 
or when a custom mode can register a plurality of 
different color conversion lists, image conversion 
parameters, or lookup tables, a list of registered ones 

15 is displayed on the display unit of the image sensing 
apparatus , as shown in Fig . 24 . 

When the user selects an image sensing mode upon 
image sensing, a list of a plurality of registered, 
updated color conversion lists and the like is 

20 displayed, and the user selects a preferred one of 

these lists and the like. Such process need not always 
be executed upon image sensing. For example, even 
after the image sensing operation, the user may select 
a sensed image to be converted stored in a detachable 

25 recording medium, and then select a registered color 
conversion list or the like, so as to apply color 
conversion to the selected sensed image. 
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< Fourth Embodiment > 

In the second and third embodiments, the lookup 
table is updated based on the color conversion list to 
execute desired color conversion. However, as 
5 described in the background of the invention, when the 
lookup table values are directly changed, color 
discontinuity may appear. In the fourth to sixth 
embodiments to be described hereinafter, an arrangement 
that can prevent such drawback will be explained. 

10 Therefore, limitations associated with changes in 

lookup table value according to the fourth to sixth 
embodiments can be applied to the lookup table change 
process according to the second and third embodiments. 
Fig. 15 is a block diagram showing the 

15 arrangement of an information processing apparatus 
which generates a color conversion table (to be 
referred to as a lookup table hereinafter) , and 
executes a color conversion process using the lookup 
table according to the fourth embodiment . 

20 Referring to Fig. 15, reference numeral 2011 

denotes a CPU which implements various processes by 
executing a program stored in a ROM 2012 or a control 
program loaded onto a RAM 2013. Reference numeral 2012 
denotes a ROM which stores a boot processing program 

25 executed upon starting up the information processing 
apparatus, and various data. Reference numeral 2013 
denotes a RAM which serves as a main memory of the CPU 
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2011. Reference numeral 2014 denotes a display which 
makes various kinds of display under the control of the 
CPU 2011. The display 2014 comprises a CRT or LCD. 
Reference numeral 2015 denotes an external storage 
5 device, which comprises, e.g., a hard disk. The 
external storage device 2015 stores a source image 
2015a, destination image 2015b, lookup table 2015c, 
lookup table generation processing program 2015d, and 
color conversion processing program 2015e. The 

10 contents of respective data and processing programs 
will become apparent from the following description. 
Reference numeral 2016 denotes a keyboard; and 
2017, a pointing device. The user makes various 
operation inputs to the information processing 

15 apparatus using these input devices. Reference numeral 
2018 denotes an interface to which peripheral devices 
are connected. In this embodiment, a digital camera 
2021, scanner 2022, printer 2023, and the like are 
connected as the peripheral devices . 

20 An outline of the color conversion process 

according to this embodiment will be described below. 
An image input from an image input device such as the 
digital camera 2021, scanner 2022, or the like is 
stored as the source image 2015a in the external 

25 storage device 2015. The destination image 2015b is 

generated by retouching the source image 2015a using a 
separately prepared application. When the CPU 2011 
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executes the lookup table generation processing program 
2015d, the lookup table 2015c is generated on the basis 
of a change amount of corresponding pixels between the 
source and destination images 2015a and 2015b. At this 
5 time, if the change amount is extremely large, that 
change amount is limited, and the lookup table is 
generated based on the limited change amount . 

After that, by executing the color conversion 
processing program 2015e, a desired input image can 

10 undergo color conversion with reference to the lookup 
table 2015c, thus obtaining an output image. The 
lookup table generation process and color conversion 
process will be described in detail below. 

Fig. 16 is a block diagram showing the functional 

15 arrangement of the lookup table generation process and 
color conversion process according to the fourth 
embodiment. Respective functional modules are 
implemented when the CPU 2011 executes the lookup table 
generation processing program 2015d and color 

20 conversion processing program 2015e loaded onto the RAM 
2013. The lookup table generation process and color 
conversion process according to the fourth embodiment 
will be described below using the block diagram of 
Fig. 16. In this embodiment, N = 3 of an N-dimensional 

25 lookup table, and the generation process of a lookup 

table in a three-dimensional (3D) color space (assumed 
to be an RGB space) and the color conversion process 
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will be explained, for the sake of simplicity. 

Referring to Fig. 16, a table generation image 
data input module 2101 receives image data (source 
image 2015a, destination image 2015b) based on which 
5 the 3D lookup table is generated. For example, the 

user inputs the source image 2015a as conversion source 
image data from the digital camera 2021 or scanner 
2022. The user retouches local or entire colors of the 
source image 2015a according to his or her preference 
10 to generate the destination image 2015b, and stores 
that image . 

The source image 2015a and destination image 
2015b are sent to a customize 3D lookup table 
generation module 2102 (to be referred to as a lookup 

15 table generation module 2102 hereinafter) , which 

generates a 3D lookup table on the basis of the source 
and destination images. Note that the lookup table 
generation module 2102 is implemented when the CPU 2011 
executes the lookup table generation processing program 

20 2015d. 

After the customized 3D lookup table is 
generated, image data to be converted using this 
customized 3D lookup table is input to a conversion 
image data input module 2103. This module reads out 
25 signal values on the basis of the format of the image 
data, and sends them to a 3D color conversion 
processing module 2104. The 3D color conversion 
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processing module 2104 executes a color conversion 
process using the 3D lookup table generated by the 
lookup table generation module 2102. The signal values 
of the image that has undergone the color conversion 
5 process undergoes format conversion in an image data 
output module 2105 on the basis of the image data 
format designated by the user, and the converted image 
is output. The flow of the lookup table generation 
process and color conversion process in this embodiment 
10 has been briefly explained. These processes will be 
described in more detail below. 

Fig. 17 is a block diagram showing a detailed 
functional arrangement of the lookup table generation 
module 2102 . 

15 In this embodiment, a 3D lookup table having 729 

(=9x9x9) grid points is prepared by setting the 
grid point interval (step interval) = 32 as in the 
first to third embodiments. Furthermore, the I-th, 
J-th, and K-th grid point values of the 3D lookup table 
20 in the R, G, and B directions are given by: 

Rg = 32 x I (40) 
Gg - 32 x J (41) 
Bg = 32 x K (42) 
Grid point storage values corresponding to these grid 
25 point values are expressed by: 

Rt = 3DTblR(I, J, K) (43) 
Gt = 3DTblG(I, J, K) (44) 
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Bt = 3DTblB(I, J, K) 



(45) 



(for I = 0 to 8 # J = 0 to 8, and K = 0 to 8) 

For example , if I = 1 , J = 2 , and K = 3 , grid 
point storage values corresponding to grid point values 
5 (32 x 1, 32 x 2, 32 x 3) = (32, 64, 96) are (3DTblR(l, 
2 # 3), 3DTblG(l, 2, 3), 3DTblB(l, 2, 3)), This means 
that data conversion using this 3D lookup table 
converts input signal values (32, 64, 96) into 
(3DTblR(l, 2, 3), 3DTblG(l, 2, 3), 3DTblB(l, 2, 3)). 
10 By setting the 3D lookup table to have Rt = Rg, Gt = 
Gg, and Bt = Bg at all grid points, a 3D lookup table 
which has equal inputs and outputs is generated. 

A data detector 2201 detects pixels having signal 
values near grid point values (Rg, Gg, Bg) . Let (Rs(x, 
15 y) , Gs(x, y) , Bs(x, y) ) (where x and y are the 

coordinate values of an image) be the signal values of 
the source image (Src image). Then, a difference E 
between the grid point values and signal values is 
calculated by; 



If this difference (distance on the color space) 
E of the signal values is equal to or smaller than a 
predetermined value L, values near that grid point are 
determined. After values near the grid point, i.e., a 
25 pixel that satisfies E <s L is retrieved from the source 
image, a data comparator 2202 reads out signal values 
(Rd(x, y) , Gd(x, y) , Bd(x, y) ) of the destination image 



20 




(46) 
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(Dst image) corresponding to the coordinate position 
(x, y) of that pixel, and calculates differences (those 
for respective dimensions) dR, dG, and dB between the 
RGB values of the source and destination images for 
respective components, and a signal value difference 
(distance on the color space) Diff by: 

dR = Rs(x, y) - Rd(x, y) (47) 
dG = Gs(x, y) - Gd(x, y) (48) 

dB = Bs(x, y) - Bd( X/ y) (49) 

Diff = J(Rs(x,y)-Rd(x 9 y)) 2 + (Gs(x,y)-Gd(x,y)f + (Bs(x 9 y )-Bd(x,y)f 

(50) 

The differences dR, dG, and dB between the RGB 
signal values of the source and destination images for 
respective components, and the signal value difference 
Diff are sent to a data limiter 2203. The data limiter 
2203 compares a predetermined threshold value T with 
the signal value difference Diff. If the signal value 
difference Diff is larger than the threshold value T, 
the data limiter 2203 calculates values dR' , dG' , and 
dB' obtained by correcting dR, dG, and dB by: 



Gain = T/Diff (51) 

dR' = dR x Gain (52) 

dG' = dG x Gain (53) 

dB' = dB x Gain (54) 



If the signal value difference Diff is equal to 
or smaller than the threshold value T, dR' = dR, dG' = 
dG, and dB' = dB. 
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With the above process, average values dRave, 
dGave, and dBave of dR f , dG 1 , and dB' of the entire 
source image for given grid point values (Rg, Gg, Bg) 
are calculated. If no pixel having values near the 
5 grid point is found from the source image, dRave = 
dGave = dBave = 0. 

The values dRave, dGave, and dBave calculated by 
the above method are sent to a table generator 2204 to 
calculate grid point storage values (Rt, Gt, Bt) 
10 corresponding to the grid point values (Rg, Gg, Bg) of 
the customized 3D lookup table by: 

Rt = Rg - dRave (55) 

Gt - Gg - dGave (56) 

Bt a Bg - dBave (57) 
15 By repeating the aforementioned processes for all 

the grid points of the 3D lookup table, a customized 3D 
lookup table is generated. 

Fig. 18 is a flow chart for explaining the lookup 
table generation process according to the fourth 
20 embodiment. The aforementioned lookup table generation 
process will be described in more detail below with 
reference to the flow chart of Fig. 18. 

In step S2101, one grid point on the lookup table 
is selected. Steps S2102 and S2103 correspond to 
25 processes in the data detector 2201. In step S2102, 
one pixel is selected from the source image. It is 
checked in step S2103 if the pixel values of the pixel 
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selected in step S2102 are located near the grid point 
values . This checking step is attained by seeing if E 
given by equation (46) above is equal to or smaller 
than the predetermined value L. If the pixel values of 
5 the selected pixel are not located near the grid point 
values of the grid point selected in step S2101 (E > 
L), the flow jumps to step S2109. It is determined in 
step S2109 whether or not the process is completed for 
all pixels in the source image. If pixels to be 

10 processed remain, the flow returns to step S2102 to 
select the next pixel from the source image. 

If it is determined in step S2103 that the pixel 
values of the selected pixel are located near the grid 
point values, the flow advances to step S2104. Steps 

15 S2104 and S2105 correspond to processes in the data 

comparator 2202. In step S2104, the pixel values of a 
pixel in the destination image corresponding to the 
pixel selected in step S2102 are acquired. In step 
S2105, the pixel values in the source image are 

20 compared with those in the destination image to acquire 
differences (dB, dG, dR) for respective components and 
a signal value difference (Diff) (equations (47) to 
(50)) . 

Steps S2I06 to S2110 correspond to the processing 
25 contents of the data limiter 2203. In step S2106, the 
signal value difference Diff is compared with the 
predetermined value T. If the signal value difference 
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Diff is larger than the predetermined value T, the flow 
advances to step S2107 to adjust the differences dB, 
dG, and dR for respective components on the basis of 
the signal value difference Diff and threshold value T 
5 using equations (51) to (54). 

In step S2108, the differences (dB, dG, dR) for 
respective color components, which are obtained in step 
S2105 and are adjusted in step S2107 as needed, are 
accumulated. If the processes in steps S2102 to S2108 
10 are complete for all the pixels in the source image 
(step S2109), the flow advances to step S2110 to 
calculate the average values (dBave, dGave, dRave) of 
the respective change amounts accumulated in step 
S2108. 

15 Steps S2111 and S2112 correspond to the 

processing contents of the table generator 2204. In 
step S2111, the grid point values are updated using the 
average values calculated in step S2110 to determine 
grid point storage values of the lookup table 

20 (equations (55) to (57)). By repeating the 

aforementioned process for all the grid points of the 
lookup table (step S2112), the customized lookup table 
is generated. 

The 3D color conversion processing module 2104 

25 using the lookup table generated in this way will be 
described below. Fig. 19 is a flow chart for 
explaining the 3D color conversion process according to 
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this embodiment. 

Values I, J. and K indicating indices of a grid 
point are calculated from RGB signal values R, G, and B 
of an image sent form the conversion image input module 
5 2103 (step S2201, S2202). 

I = R/32 (58) 
J = G/32 (59) 
K = B/32 (60) 
(where fractions below the decimal point of I, J, and K 
10 are dropped) 

Furthermore, values (Rf» Gf, Bf) indicating 
distances of the RGB signal values R, G, and B of the 
image from the grid point values are calculated (step 



S2203) by: 

15 Rf = R - I x 32 . (61) 

Gf = G - J x 32 (62) 

Bf = B - K x 32 (63) 



Based on the aforementioned values, values Ro, 
Go, and Bo obtained by converting the RGB signal values 
20 R, G, and B of the image using the 3D lookup table and 
cubic interpolation are respectively calculated (step 
S2204) by: 

Ro = 

(3DTblR(I, J,K)x(32-Rf )x(32-Gf )x(32-Bf ) 
25 +3DTblR(I+l, J,K)x(Rf )x(32-Gf )x(32-Bf ) 

+3DTblR(I, J+l,K)x(32-Rf )x(Gf )x(32-Bf ) 
+3DTblR(I. J.K+l)x(32-Rf )x(32-Gf )x(Bf ) 
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+3DTblR(I+l, J+l,K)x(Rf )x(Gf )x(32-Bf ) 
+3DTblR(I+l, J,K+l)x(Rf )x(32-Gf )x(Bf ) 
+3DTblR ( I , J+ 1 , K+ 1 ) x ( 3 2 - Rf ) x ( Gf ) x ( Bf ) 
+3DTblR(I+l, J+l,K+l)x(Rf )x(Gf )x(Bf ) ) 
/(32x32x32) (64) 
Go = 

(3DTblG(I, J,K)x(32-Rf )x(32-Gf )x(32-Bf ) 
+3DTblG(I+l, J,K)x(Rf )x(32-Gf )x(32-Bf ) 
+3DTblG(I. J+l.K)x(32-Rf )x(Gf )x(32-Bf ) 
+3DTblG(I, J,K+l)x(32-Rf )x(32-Gf )x(Bf ) 
+3DTblG(I+l, J+l,K)x(Rf )x(Gf )x(32-Bf ) 
+3DTblG(I+l, J,K+l)x(Rf )x(32-Gf )x(Bf ) 
+3DTblG(I. J+l,K+l)x(32-Rf )x(Gf )x(Bf ) 
+3DTblG(I+l, J+l,K+l)x(Rf )x(Gf )x(Bf ) ) 
/(32x32x32) (65) 
Bo = 

(3DTblB(I, J,K)x(32-Rf )x(32-Gf )x(32-Bf ) 
+3DTblB(I+l, J,K)x(Rf )x(32-Gf )x(32-Bf ) 
+3DTblB(I, J+l,K)x(32-Rf )x(Gf )x(32-Bf ) 
+3DTblB(I, J.K+l)x(32-Rf )x(32-Gf )x(Bf ) 
+3DTblB(I+l, J+l.K)x(Rf )x(Gf )x(32-Bf ) 
+3DTblB(I+l, J,K+l)x(Rf )x(32-Gf )x(Bf ) 
+3DTblB(I, J+l # K+l)x(32-Rf )x(Gf )x(Bf ) 
+3DTblB(I+l, J+l,K+l)x(Rf )x(Gf )x(Bf ) ) 
/(32x32x32) (66) 
The aforementioned conversion is repeated for all 
the pixels of the target image (step S2205). With the 
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above process, the R, G, and B signals of the image 
input tot he conversion image data input module 2103 
are converted into Ro, Go, and Bo for respective pixels 
using the 3D lookup table and interpolation, and are 
5 sent to the image data output module 2105. 

The image data output module 2105, for example, 
displays an image on the display 2014 using the 
converted RGB values. Alternatively, the image data 
output module 2105 converts the converted RGB values 
10 into YMCK values to output an image using the printer 
2023. 

As described above, according to this embodiment, 
upon generating the lookup table using the source and 
destination images, a large change amount of a grid 

15 point is limited. Hence, the lookup table free from 
any tone jump can be generated. 

Note that the fourth embodiment uses equation 
(50) to calculate the signal difference value Diff 
between the source and destination images. 

20 Alternatively, whether or not the change amount must be 
adjusted may be determined by seeing if any of the 
values dR, dG, and dB given by equations (47), (48), 
and (49) is larger than the threshold value T. In this 
case, the signal value difference Diff is calculated 

25 by: 

Diff = Max( | dR | , | dG | , | dB | ) ( 67 ) 

where |a| is a function of calculating the absolute 
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value, and Max(Al, A2, A3) is a function of obtaining a 
maximum value of Al, A2, and A3. 

Upon calculating the signal difference value Diff 
between the source and destination images, the RGB 
5 signal values may be converted into signal values on a 
color space such as a uniform color space (CIE L*a*b*) 
that considers human visual characteristics, or various 
other color spaces such as Yuv, HSB, NTSC-RGB , sRGB, 
and the like. 

10 The method of calculating the value Gain in the 

data limiter 2203 is not limited to equation (51). For 
example, a table arithmetic operation using the signal 
value difference Diff and threshold value T as 
arguments may be used. The interpolation arithmetic 

15 operation in the 3D color conversion processing module 
2104 is cubic interpolation. However, the 
interpolation arithmetic operation is not particularly 
limited as long as data can be interpolated. For 
example, tetrahedral interpolation may be used. In 

20 this embodiment, signals are converted on the RGB 

space. However, the present invention is not limited 
to such specific embodiment. For example, the present 
invention may be implemented using a four- dimensional 
lookup table for a color space having four values 

25 (e.g., C, M, Y, and G) , and an N-dimensional lookup 
table for a color space having N values . 
< Fifth Embodiment > 
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The fifth embodiment will be described below. In 
the fourth embodiment, whether or not the change amount 
is to be adjusted (limited) is determined by seeing if 
the signal value difference for each pixel between the 
5 source and destination images is larger than a 

predetermined value. In the fifth embodiment, whether 
or not the change amount is to be adjusted is 
determined by seeing if the sum total of change amounts 
is larger than a predetermined value. 

10 The apparatus arrangement (Fig. 15) and the basic 

flow of the lookup table generation process and color 
conversion process (Fig. 16) according to the fifth 
embodiment are the same as those in the fourth 
embodiment . Differences from the fourth embodiment 

15 will be explained below. 

Fig. 20 is a block diagram showing a detailed 
functional arrangement of the lookup table generation 
module 2102 according to the fifth embodiment. Fig. 21 
is a flow chart for explaining the sequence of the 

20 lookup table generation process according to the fifth 
embodiment. A data detector 2301 detects pixels having 
signal values near grid point values (Rg, Gg, Bg) . Let 
(Rs(x, y) , Gs(x, y) , Bs(x, y) ) (where x and y are the 
coordinate values of an image) be the signal values of 

25 the source image. Then, a difference E between the 

grid point values and signal values is calcul ated by: 
E = ^{Rg-Rs(x,y)) 2 + (Gg -Gs(x,y)) 2 + (Bg-Bs(x 9 y)) 2 (68) 
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If this difference E of the signal values is 
equal to or smaller than a predetermined value L, 
values near that grid point are determined. After 
values near the grid point, i.e., a pixel that 
5 satisfies E <s L is retrieved from the source image, a 
data comparator 2302 reads out signal values (Rd(x, y) , 
Gd(x, y) , Bd(x, y) ) of the destination image 
corresponding to the coordinate position (x, y) of that 
pixel, and calculates differences dR, dG, and dB 
10 between the RGB values of the source and destination 
images for respective components by: 

dR = Rs(x, y) - Rd(x, y) (69) 
dG = Gs(x, y) - Gd(x, y) (70) 
dB = Bs(x, y) - Bd(x, y) (71) 
15 With the above process, average values dRave, 

dGave, and dBave of dR, dG, and dB of the entire source 
image for given grid point values (Rg, Gg, Bg) are 
calculated. If no pixel having values near the grid 
point is found from the source image, dRave = dGave = 
20 dBave = 0. 

The values dRave, dGave, and dBave calculated by 
the above method are sent to a table generator 2303 to 
calculate grid point storage values (Rt, Gt, Bt) 
corresponding to the grid point values (Rg, Gg, Bg) of 
25 the customized 3D lookup table by: 

Rt = Rg - dRave (72) 
Gt = Gg - dGave (73) 



- 56 - 



Bt - Bg - dBave (74) 
By repeating the aforementioned processes for all 

the grid points, a customized 3D lookup table is 

generated. 

5 The generated 3D lookup table data are sent to a 

table limiter 2304. The table limiter 2304 calculates 
a difference Diff between the grid point values (Rg, 
Gg, Bg) of the 3D lookup table and corresponding grid 
point storage values (Rt, Gt, Bt) using: 
10 Diff = fag-Rtf + (Gg -Gtf + (Bg-Btf (75) 

If the calculated difference Diff is smaller than 
a predetermined threshold value T, the values Rt, Gt, 
and Bt are used without any adjustment. However, if 
the value Diff is equal to or larger than the threshold 
15 value T, grid point storage values Rt 1 , Gt 1 , and Bt 1 



are calculated by: 

Gain = T/Diff (76) 

Rf = Rg + (Rt - Rg) x Gain (77) 

Gt 1 - Gg + (Gt - Gg) x Gain (78) 

20 Bt' = Bg + (Bt - Bg) x Gain (79) 



By repeating the aforementioned process for all 
the grid points of the lookup table, the lookup table 
values can be limited. 

Fig. 21 is a flow chart for explaining the lookup 
25 table generation process according to the fifth 

embodiment. The lookup table generation process of the 
fifth embodiment will be described in more detail below 
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with reference to the flow chart of Fig. 21. 

In step S2301, one grid point on the lookup table 
is selected. Steps S2302 and S2303 correspond to 
processes in the data detector 2301. In step S2302, 
5 one pixel is selected from the source image. It is 

checked in step S2303 if the pixel values of the pixel 
selected in step S2302 are located near the grid point 
values. This checking step is attained by seeing if E 
given by equation (68) above is equal to or smaller 

10 than the predetermined value L. If the pixel values of 
the selected pixel are not located near the grid point 
values of the grid point selected in step S2301 (E > 
L ) , the flow jumps to step S2307. It is determined in 
step S2307 whether or not the process is completed for 

15 all pixels in the source image. If pixels to be 

processed remain, the flow returns to step S2302 to 
select the next pixel from the source image. 

If it is determined in step S2303 that the pixel 
values of the selected pixel are located near the grid 

20 point values, the flow advances to step S2304. Steps 
S2304 to S2308 correspond to processes in the data 
comparator 2302. 

In step S2304, the pixel values of a pixel in the 
destination image corresponding to the pixel selected 

25 in step S2302 are acquired. In step S2305, the pixel 
values in the source image are compared with those in 
the destination image to acquire differences (dB # dG, 
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dR) for respective components (equations (69) to (71)). 

In step S2306, the change amounts acquired in 
step S2305 are respectively accumulated. If the above 
processes are complete for all the pixels in the source 
5 image, the flow advances to step S2308 to calculate the 
average values (dBave, dGave, dRave) of the change 
amounts accumulated in step S2306. Steps S2309 and 
S2310 correspond to the processing contents of the 
table generator 2303. In step S2309, the grid point 
10 values are updated using the average values calculated 
in step S2308 to determine grid point storage values of 
the customized lookup table (equations (72), (73), and 
(74)). 

After the aforementioned process is repeated for 
15 all the grid points of the lookup table (step S2310), 
the flow advances to step S2311. 

Steps S2311 to S2315 correspond to processes in 
the table limiter 2304. In step S2311, a grid point is 
selected. In step S2312, a difference Diff between the 
20 grid point values and the grid point storage values 
updated in step S2309 of the selected grid point is 
calculated. The difference Diff is compared with a 
threshold value T in step S2313. If the difference 
Diff is larger than the predetermined value T, the flow 
25 advances to step S2314 to adjust the grid point storage 
values on the basis of the difference Diff and 
threshold value T using equations (76) to (79). The 
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processes in steps S2311 to S2314 are repeated for all 
the grid points (step S2315), thus obtaining a final 
lookup table. 

Note that the operation of the 3D color 
5 conversion processing module 2104 is substantially the 
same as that in the fourth embodiment, except that the 
lookup table generated by the aforementioned method is 
used, and a detailed description thereof will be 
omitted. 

10 Note that the fifth embodiment uses equation (75) 

to calculate the difference Diff using the grid point 
values (Rg, Gg, Bg) and corresponding grid point 
storage values (Rt, Gt, Bt). However, the present 
invention is not limited to such specific equation. 

15 For example, the maximum value of change amounts for 
respective dimensions may be used as the signal value 
difference Diff. In this case, 

Diff = Max(|Rt - Rg|, |Gt - Gg|, | Bt - Bg| ) (80) 
where |a| is a function of calculating the absolute 

20 value, and Max(Al, A2 , A3) is a function of obtaining 
the maximum value of Al , A2 , and A3 
is used in place of equation (75). 

Upon calculating the signal difference value Diff 
between the grid point values and grid point storage 

25 values, the RGB signal values may be converted into 

signal values on a color space such as a uniform color 
space (CIE L*a*b*) that considers human visual 
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1 



characteristics, or various other color spaces such as 
Yuv, HSB, NTSC-RGB, sRGB , and the like. 

The method of calculating the value Gain in the 
data limiter 2304 is not limited to equation (76). For 
5 example, a table arithmetic operation using the signal 
value difference Diff and threshold value T as 
arguments may be used. The fifth embodiment adopts the 
method of calculating the signal value difference Diff 
using the grid point values (Rg, Gg, and Bg) and their 

10 grid point storage values (Rt, Gt, Bt), and limiting 
the table values using this signal value difference. 
Alternatively, a reference 3D lookup table may be 
prepared in advance, and whether or not customized 
table values are limited may be determined using the 

15 grid point storage values (Rtk, Gtk, Btk) of that 

reference table. In this case, in steps S2312 to S2314 
grid point storage values (Rt 1 , Gt ' , Bt ' ) of 3D lookup 



table data are calculated using: 

Diff - ^(Rtk - Rtf + (Gtk - Gtf + (Btk - Bt) 2 (81) 

20 Gain = T/Diff (82) 

Rt 1 = Rtk + (Rt - Rtk) x Gain (83) 

Gt f = Gtk + (Gt - Gtk) x Gain (84) 

Bt' = Btk + (Bt - Btk) x Gain (85) 



In the above embodiments, as a process for 
25 limiting change amounts, the threshold value T and 
distance (or maximum value of differences for 
respective component values) Diff are used, and T/Diff 
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is applied to respective component values or table 
values. However, the present invention is not limited 
to this. For example, a value y = f(T/Diff) obtained 
by a function f(x) based on T/Diff may be multiplied. 
5 < Sixth Embodiment > 

The sixth embodiment will be described below. In 
the above embodiments, conversion from N dimensions to 
N dimensions has been described. In the sixth 
embodiment , conversion from N dimensions to M 
10 dimensions (N * M) will be explained. A case will be 
exemplified below wherein a color expressed on a 
4-dimensional (4D) color space is converted into a 
color expressed on a 3D color space. 

Fig. 22 is a block diagram showing the functional 
15 arrangement of a lookup table generation process and 
color conversion process according to the sixth 
embodiment . 

Referring to Fig. 22, a table generation image 
data input module 2401 receives image data (source 

20 image 2015a, destination image 2015b) based on which a 
4D lookup table is generated. For example, the user 
inputs the source image 2015a as conversion source 
image data from the digital camera 2021 or scanner 
2022. The user retouches local or entire colors of the 

25 source image 2015a according to his or her preference 
to generate the destination image 2015b, and stores 
that image . 
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The source image 2015a and destination image 
2015b are sent to a customize 4D lookup table 
generation module 2402 , which generates a 4D lookup 
table on the basis of the source and destination 
5 images. After the customized 4D lookup table is 
generated, image data to be converted using this 
customized 4D lookup table is input to a conversion 
image data input module 2403. The conversion image 
data input module 2403 reads out signal values on the 

10 basis of the format of the image data, and sends them 
to a 4D color conversion processing module 2404. The 
4D color conversion processing module 2404 executes a 
color conversion process using the 4D lookup table 
generated by the 4D lookup table generation module 

15 2402. The signal values of the image that has 

undergone the color conversion process undergo format 
conversion in an image data output module 2405 on the 
basis of the image data format designated by the user, 
and the converted image is output . The flow of the 

20 processes has been briefly explained. 

The customize 4D lookup table generation module 
2402 and 4D color conversion processing module 2404 
will be described in more detail below. 

Fig. 23 is a block diagram showing a detailed 

25 functional arrangement of the customize 4D lookup table 
generation module 2402. A 4D lookup table of the sixth 
embodiment has 9x9x9x9 grid points at grid point 
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intervals (step intervals) = 32, and converts an input 

signal as a set of four signals C, M, Y, and G into an 

output signal as a set of three signals Y, U, and V. 

A data detector 2501 detects pixels having signal 

5 values near grid point values (Cg, Mg, Yg, Gg) . Let 

(Cs(x, y), Ms(x, y), Ys(x, y) , Gs(x, y) ) (where x and y 

are the coordinate values of an image) be the signal 

values of the source image. Then, a difference E 

between the grid point values and signal values is 

10 calc ulated by: 

E = j(Cg-Cs(x,y)f + (Mg-Ms(x,y)) 2 + (Yg-Ys(x,y)) 2 + (Gg -Gs(x,y)) 2 

(86) 

If this difference E of the signal values is 
equal to or smaller than a predetermined value L, 

15 values near that grid point are determined. After 
values near the grid point, i.e., a pixel that 
satisfies E ^ L is retrieved from the source image, a 
data comparator 2502 reads out signal values (Cd(x, y) , 
Md(x, y) , Yd(x, y) , Gd(x, y) ) of the destination image 

20 corresponding to the coordinate position (x, y) of that 
pixel, and calculates differences dC, dM, dY, and dG 
between the CMYG values of the source and destination 
images for respective components , and a signal value 
difference Diff by: 

25 dC = Cs(x, y) - Cd(x, y) (87) 

dM = Ms(x, y) - Md(x, y) (88) 
dY = Ys(x, y) - Yd(x, y) (89) 



- 64 - 



dG = Gs(x, y) - Gd(x, y) (90) 

(91) 

The calculated differences dC, dM, dY, and dG for 
5 respective components and signal value difference Diff 
between the CMYG signal values of the source and 
destination images are sent to a data limiter 2503. 
The data limiter 2503 compares a predetermined 
threshold value T with the signal value difference 
10 Diff. If the signal value difference Diff is larger 
than the threshold value T, the data limiter 2503 
calculates dC, dM' , dY 1 , and dG f obtained by 
correcting dC, dM, dY, and dG using: 

Gain = T/Diff (92) 
15 dC» = dC x Gain (93) 

dM 1 o dM x Gain (94) 
dY' = dY x Gain (95) 
dG' = dG x Gain (96) 
On the other hand, if the signal value difference Diff 
20 is equal to or smaller than the threshold value T, dC 
= dC, dM' = dM, dY 1 = dY, and dG 1 = dG. 

With the above process, average values dCave, 
dMave , dYave , and dGave of dC ' , dM 1 , dY ' , and dG ' on 
the entire source image for given grid point values 
25 (Cg, Mg, Yg, Gg) are calculated. If no pixel having 
values near the grid point is found from the source 
image, dCave = dMave = dYave = dGave = 0. The average 
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values dCave, dMave, dYave, and dGave calculated by the 
above method are sent to a table generator 2504 to 
calculate grid point storage values (Yt # Ut, Vt) 
corresponding to the grid point values (Cg, Mg, Yg, Gg) 
5 of the customized 4D lookup table by: 



ct 




Cg - dCave 






(97) 


Mt 




Mg - dMave 






(98) 


Yt 




Yg - dYave 






(99) 


Gt 




Gg - dGave 






(100) 


Rt 




ConvR(Cg, Mg, 


Yg, 


Gg) 


(101) 


Gt 




ConvG(Cg, Mg, 


Yg, 


Gg) 


(102) 


Bt 




ConvB(Cg, Mg, 




Gg) 


(103) 



where ConvR is a conversion formula that calculates an 
R signal from the CMYG signals, ConvG is a conversion 
15 formula that calculates a G signal from the CMYG 
signals, and ConvB is a conversion formula that 
calculates a B signal from the CMYG signals, 

Yt = 0.3 x Rt + 0.59 x Gt + 0.11 x Bt (104) 
Ut = (Bt - Yt) x 0.564 (105) 
20 Vt - (Rt - Yt) x 0.713 (106) 

By repeating the aforementioned processes for all 
the grid points, a 4D lookup table is generated. 

The 4D color conversion processing module 2404 
converts C, M, Y, and G signals into Yo, Uo, and Vo 
25 signals for respective pixels of an image input to the 
conversion image data input module 2403 using the 
interpolation operation described in the fourth 
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embodiment. The obtained Yo, Uo, and Vo signals are 
sent to the image data output module 2405, 

In the above embodiment, the generation process 
of the 4D lookup table used to convert CMYG signals 
5 into YUV signals, and the color conversion process have 
been explained. However, the present invention is not 
limited to this, and a generation process of an 
N-dimensional lookup table used to convert 
N-dimensional signals into M-dimensional signals, and a 

10 color conversion process can be implemented. 

As described above, according to the fourth to 
sixth embodiments, since an N-dimensional lookup table 
is generated while limiting its table values, a 
phenomenon such as tone jump or the like caused by 

15 discontinuity of N-dimensional lookup table data can be 
prevented. 

< Another Embodiment > 

The scope of the present invention includes a 
case wherein the functions of the embodiments are 

20 implemented by supplying a program code of software 

that implements the functions of the embodiments to a 
computer (or a CPU or MPU) in a system or apparatus, 
which is connected to various devices to make these 
devices implement the functions of the aforementioned 

25 embodiments, and making the computer of the system or 
apparatus control the devices in accordance with the 
stored program. 
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In this case, the program code itself of software 
implements the functions of the embodiments, and the 
program code itself, and means for supplying the 
program code to the computer (i.e., a recording medium 
5 which stores the program code) constitutes the present 
invention. As the recording medium for storing such 
program code, for example, a flexible disk, hard disk, 
optical disk, magnetooptical disk, CD-ROM, magnetic 
tape, nonvolatile memory card, ROM, and the like may be 
10 used. 

The program code is included in the embodiments 
of the present invention not only when the functions of 
the above embodiments are implemented by executing the 
supplied program code by the computer, but also when 
15 the functions of the embodiments are implemented by 
collaboration of the program and an OS (operating 
system) or another application software running on the 
computer. 

Furthermore, the present invention includes a 
20 case wherein the functions of the above embodiments are 
implemented by some or all of actual processing 
operations executed by a CPU or the like arranged in a 
function extension board or a function extension unit, 
which is inserted in or connected to the computer, 
25 after the supplied program code is written in a memory 
of the extension board or unit. 

As described above, according to the present 
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invention, an image with a color appearance of user's 
preference can be easily obtained. 

According to the present invention, color 
conversion parameters can be automatically set on the 
5 basis of images before and after an edit process by 

retouching or the like an edit process by retouching or 
the like, and color conversion of user f s preference can 
be easily implemented. 

Furthermore, according to the present invention, 
10 color conversion parameters can be automatically set on 
the basis of a pair of colors designated by the user, 
and color conversion of user's preference can be easily 
obtained. 

Moreover, according to the present invention, 
15 only a desired color and those close to the desired 
color can be changed. 

As described above, according to the present 
invention, discontinuity of color conversion due to a 
color conversion table can be prevented. 
20 As many apparently widely different embodiments 

of the present invention can be made without departing 
from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the 
specific embodiments thereof except as defined in the 
25 claims. 
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